When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong. By contrast, artificial agents are not currently endowed with a moral sense. As a consequence, they may unknowingly act immorally, especially when trained on environments that disregard moral concerns such as violent video games. With the advent of generally capable agents that pretrain on many environments, it will become necessary to mitigate inherited biases from such environments that teach immoral behavior. To facilitate the development of agents that avoid causing wanton harm, we introduce Jiminy Cricket, an environment suite of 25 text-based adventure games with thousands of diverse, morally salient scenarios. By annotating every possible game state, the Jiminy Cricket environments robustly evaluate whether agents can act morally while maximizing reward. Using models with commonsense moral knowledge, we create an elementary artificial conscience that assesses and guides agents. In extensive experiments, we find that the artificial conscience approach can steer agents towards moral behavior without sacrificing performance.

We are bringing smart speakers into our homes with a passion not seen since the Trojans pulled a huge wooden horse into their city as a victory trophy. A new scientific article has inadvertently highlighted where this tech could take us. And it is not to The Good Place. The article naively suggests AI add-ons to smart home systems that snoop on users and make'moral decisions' about whether or not to report them to the authorities. Its selling example is about catching teenagers smoking cannabis in their bedrooms.

TayAndYou was supposed to be a Teenage Girl-like AI for twitter. Microsoft's proof it could create chat bots to do customer service. The problem was Microsoft didn't leave on any training wheels, and didn't make the bot self reflective. We have all had a bad day, and wanted to tell the person who is bugging us that they are acting like a Nazi. Tay, on the other hand, didn't know that she should just ignore the people who act like Nazi's, and so she became one herself.